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A study program and test collectSxjn project was 
recently completed/ by the Clearinghouse for AppliW Performance 
Testing at the Northwest Regional Educational Laboratory (NWBEL) . The 
project focused on two separate, but related, objectives: compilation 
of self-study programs leading directly to certification or licensing 
and development of a survey of occupations to provide both general 
and specific information on licensing and certification procedures 
for those occupations in which vocational competence is determined 
through examination. To determine the availability of certification 
tests, KWBEL staff . contacted over 1,000 potential information sources 
by telephone, first-class mail of personal interview,^ University and 
municipal libraries provided 'available written references, while 
three computer searches supplied abstracts of significant educational 
and vocational articles, /Overall, the information solicitation phase 
was judged successful. Adequate information was available on all 
occupations and programs NWBEL wished to^inclu^de in the compilation, 
and the information received appears to cover the majority of 
existing occupational examination processes. (Author/HV) 
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BACKGROUND fNFORMATION^ ^ X 

A Study program and test collection project was recently completed 
by the Clearinghouse for Applied Performance Testing at the. Northwest 
Regional Educational Laboratory (NWREL) . This work was sponsored by the 
United States Office of Education at the request of the Defense Activity 
for Non -Traditional Educational Support (DANTES) . 

The project focused on two separate but related, objectives: 
compilation of self-study programs leading directly to certification or -'i 
licensing and development of a survey pf occupations to provide both 
general and specific information on licensing and certification procedures 
for those occupations in which vocational competence is determined 
through examination. 

PROCEDURES 

Information Solicitation and Collection ^ 

To determine the availability of certification tests, KWREL staff 
contacted over 1,000 potential information sources by telephone, first- 
'^class mail or personal interview. University and municipal libraries 
provided available written references, while three computer searches 
supplied abstracts of significant educational and vocational articles. 

The telephone information search yielded mixed results. In some 
cases, project personnfel were already aware of the existence of. an 
appropriate course or examination procedure, and the telephone .call was 
merely^ a^ request for more detailed information. Phone calls in such 
cases were quite effective. But when a call was made to determine the 
existence of an appropriate program, it often proved less helpful. 

Calls to state and municipal government agencies were particularly 
ineffective, possibly because of the large size of those agencies and 
the extensive division of responsibility, among agency st^aff. On the 
other hand, mo^t professional organizations and Federal agencies had 
specific, well-established programs and were there foi:e capable of 
providing adequate, complete information with minimal delay and !j:onfusion. 

At the start of the project, reference materials were needed to 
provide information for the occupational survey entries. In addition, 
it was necessary to determine what previous work had been done in 
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relevant fields- Two federal publications of obvious value — The 
Dictionary of Occupational Titles ^ and the Military Civilan Job 
Comparability llantial- ^^ — provided information useful in preparing the 
compilations* However, The Occupational Outlook Handbook-^ was the 
project's most useful reference. Its entries not only contained 
descriptions of the nature of the work and training requirements for 
each occupation, but also listed sources of additional information. 
Project staff used these listings to check the completeness of 
information they collected. The Handbook often provided general 
information which was aot immediately available from other sources. 

A few additional references were* used . The Diet ionary of 
Professional and Occupational Licensing in the United States^ and 
Equivalency and Prof iciency^ Testing^ were useful references against 
which the completeness of the DANTES occupational survey could be 
checked. Other references provided addresses for organizations, state 
agencies and unions. ^> 7, 8, 9 Occupational Licensing: Principles and 
Practices^ O offered an indepth study of current licensing concepts and 
processes. No other references were found which provided information 
directly related to project development goals. 

Computer literature searches were conducted by NWREL Infomation 
CentOT personnel using the ERIC, AIM (Abstracts of Instruction Materials) 
and ARM (Abstracts of Research Materials) systems. Several hundred 
references and their abstracts were obtained through these searches, 
but only a small percentage of the titles were appropriate, and the 
large majority of those provided only minimal information. Most- 
references related to apprenticeship cr manpower programs, or to 
individualized vocational programs in the educational setting. In 
summary, the computer searches were useful only in providing background 
information. * , 

The project's mailing program was the most valuable method of 
collecting information. A form letter was sent to a wide variety of 
agencies, organizations, and individuals who might have information to 
contribute. Information on both independent study courses leading to 
licensure or certification and the licensure or certification procedures 
themselves was requested. 

Project staff took special care to develop a mailing list which 
thoroughly covered the broad spectrum of potential information sources. 
The original mailing list selection procedures are outlined below. 

1, State Governmental Agencies . For each state, approximately five 
agencies who might have relevant information were selected^ 
from the 1974-75 National Directory of State Agencies . 7 
Selections typically included Departments of Education, 
Commerce, Labor, Consumer Affairs, and so forth. Slightly over 
300 contacts -were made through this procedure. 
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2. Other Goveriflnental Agencies * Several municipal and federal 
agencies were contacted. In a few cases the intent was to 
gather general information, but more often it was to obtain 
information. about specific known procedures. It was for this 
latter purpose that the licensing divisions of ^ the Federal 
Aviation Administration, Federal Communications Commission, 
and the United States Coast Guard were contacted. Fewer than 
ten agencies were contacted within this group. 

3. Home Study Schools > All member schools of the National Home 
Study Council were contacted. (Letters were not sent to 
schools listed as subsidiaries or divisions, but in all cases 
the parent organization was contacted.) Other correspondence 
schools listed in Lovejoy's Career and Vocational School 
Guide^ were also contacted. In all, 100 schools received the 
inf oxrmation request. 

4. Colleges and Universities . All of the approximately 75 member 
schools of the National University Extension Association (NUEA) 

^received the information solicitation. 

5. Companies . A request for information was sent to each of the 
100 largest United States industrial corporations specified in 
fche appropriate 1975 Fortune Magazine Double 500 listing. 

6. Labor ,Orpanizations . Through examination of the 1974 National 
Trade and Professional Association's Yearbook^ the names of 
approximately 70 labor unions were obtained; organizations 
which were, from name, presumed to be labor unions were 
selected from a listing of groups reporting annual budgets of 
over $1,000,000. " 

7. Professional and Trade Organizations . This category originally 
contained 70 organizations which were selected because they 
seemed potential sources of relevant information. The selec- 
tion was made on the basis of information from the Occupational 
Outlook Handbook- ^, and Lovejoy's Career and V ocatio nal School 
Guide^ as well as prior knowledge of the project staff. 

8. Other Groups and Individuals > The solicitation letter was also 
mailed to a small number of potential contributors who did not 
fall into one of the groups listed above. Names of such 
organizations and individuals were often suggested during the 
course of phone calls or were listed in relevant references 
and materials. These cpntac ts^-less than 15 in all — included 
professional educators knowledgeable in the vocational licen- 
sing field and a small number, of centers for vocational 
education or research.. 
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The form letter used to solicit information asked that the 
recipient provide project staff with names of additional contacts who 
might provide relevant information. Many did so, providing NWREL with 
over 100 additional information sources. It was the policy of projeqt— 
staff to followi^up on all suggested contacts — occasionally by phone, 
but more frequently through use of the same solicitation letter. State 
agencies most often provided these followup contacts; when the original 
agency was an inappropriate' source of inf onp.ation, their response 
usually contained the suggestion of a more probable source. Frequently 
the state agency itself forwarded NWREL^s letter to a different govern- 
ment department. 

A few individuals and universities and several professional 
organizations provided suggestions for additional infoirmation sources. 
Responses from state governmental agencies and from professional and 
trade organizations were generally supportive of the project objectives; 
therefore, such organizations often attempted to provide sources of 
information when they themselves were uroable to help. 

Table 1 summarizes the scope and success of the information 
solicitation phase in each of the classif ications'described above. 

In general, information provided by state governments was wide 
ranging^ but seldom detailed enough to be of significant value. There 
were exS^ptions, however, as some states had previously compiled single 
sources! of complete information on licensed occupations within the state. 

Most information provided by colleges and universxtxes took the form 
of printed extension service catalogs and bulletins; this information 
was detailed but not particularly applicable. Their response rate was 
moderate. Few companies or labor organizations responded and those 
which did provided no important information. Response from home study 
schools was also very low. 

The few nori-^state government agencies contacted responded well to 
specific information requests. Professional and trade organizations 
were usually able to either provide useful information and appropriate 
information sources, or state that the information requested did not 
exist. Their response rate was fairly high, and the detailed booklets, 
pamphlets and brochures provided by those organizations were instrumental 
in providing both occupational and examination information. 

^Overall, the information solicitation phase was judged successful. 
Adequate information was available on all occupations and programs NWREL 
wished to include in th^ compilation, and the information received 
appears to cover the majority of existing occupational examination 
processes. 
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Table 1 



MAILING LIST CONSTRUCTION AND RESPONSE 



Classification 



Original* 
Contacts 



Subsequent*, 
Contacts 



Total* 
Contacts 



Total** 
Responses 



State Governmental 
Agencies 300 

Other Governmental 
Agencies 5 

Home Study Schools 100 

Colleges and 

Universities 75 

Companies 100 
Labor 

Organizations 70 

Professional' and 

Trade Organizations 70 

other Groups and 
Individuals , 10 



75 

0 

0 

5 
0 



30 



375 
5 

100 

80 
100 

70 

100 

15 



175 
5 

s 

25 
5 



50 



TOTAL. 



730 



115 



845 



275 



* All figures are approximated slightly. 

# This category tabulates all written responses received, including 
materials, information, suggestions for additional contacts, and 
statements that information was not available. 
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PROJECT FINDINGS 

Evaluation of the Information Collected 

*Rie results of the information search permit some observations 
con^ning licensing and certification procedures and'related independent 
stu** courses. These observations are supported by the thorough nature 
of the project^s information collection process; however, until verified 
by widespread empirical research, they must be regarded as tentative. 

Numerous independent study courses, covering a wide range of- subject 
areas, are available. Proprietary schools, colleges, universl^es, 
junior and community colleges, and various professional and tftade 
organizations all offer self-study, correspondence courseworlc. Courses 
range from elementary through post-graduate levels, and cover everything 
from highly practical skills like welding to theoretical science and math. 
Project staff readily identified several thousand such courses. 

Yet very few of these courses lead to recognition of achievement 
through award of certificate, license, or academic degree. Most indi- 
vidual courses are apparently intended to increase knowledge or skill in 
one restricted area; seldom are, they coordinated to form a total 
"program." Project staff therefore realized that very few independent 
study courses were suitable for inclusion in a catalog of self--study 
programs — particularly a catalog of programs providing preparation for 
certification or licensing examinations. A few program.s offer 
"preparation courses" for real estate, accounting, engineering, or other 
professional examinations. In addition, the American Watchmaker's 
Institute has a self -study course and ^related certification test in 
watchmaking and a few proprietary schools have curricula which lead to 
various licenses — notably Federal Communications Commission permits. 
Yet it would probably be correct to state, if our sampling is represen- 
tative, that not one of a hundred independent study courses is directly 
related to a licensing or certification examination. 

Colleges and universities provided a large part of the independent 
study information. Even so, their response rate was not as great as 
anticipated. Even more surprising was the very sm^ll response received 
from home study schools. Of more than a hundred schools contacted only 
five responded. Clearly home study schools have materials available for 
dissemination. Therefor^, project staff concluded that either the 
schools offered no courses appropriate for inclusion in the survey, or 
perhaps the compilation .procedures , the sponsoring body, or some other 
aspect of the project made program evaluation unattractive to the 
majority of these organizations. The DANTES decision not to include 
proprietary schools in the compilation eliminated the problem of 
obtaining information from these sources. 

\ .' . . . 

Project staff were considerably more successful in compiling 
examination information. Although state governmental agencies were 
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seldom able to provide information about tests with nationwide 
applicability, these agencies were often able to supply fairly detailed 
information on requirements for a few occupations within a particular 
state (e.g., plumbers in Washington, elevator inspectors in Maine). 
Though they were usually unable to generalize licensing requirements to 
other states, that is not to say that the contacts were not useful. 
The large volume of responses of ten made it possible for project staff 
to make required generalizations^ themselves . As previously mentioned, 
among the most helpful contributions were the compilations received from 
about ten states. These documents were very valuable in developing 
survey entries on the licensed occupations. 

Companies and labor unions provided almost no input to the 
compilation; the few that did respond offered no relevant information. 
However, various supplemental readings and interviews suggested that 
most company programs were actually not appropriate for the survey in 
any event, since they were not available nationwide, nor were they 
available to people outside the employ of a specific company. Labor 
unions could have provided much information regarding how union member- 
ship and journeyman status constitute a type of certification — a concept 
which affects sejt^ral million workers in union-oriented trades. However, 
such information was outside the focus of the compilation. The response 
rate from companies and unions was so low that it is logical to conclude 
that many requests did not reach appropriate sources in the first plac^ 
(because of inadequate addresses ) , or that none of these sources had 
any pertinent information. 

Most useful information came from the various professional 
organizations, who provided all certification and a substantial amount • 
of licensing information used in the compilation. 

Comments on Licensing and Certification 

Many occupations are in some way rdgulated by state, federal, or 
local governmental agencies; such regulation is generally termed 
licensing. Licensing requirements most often originate through specific 
legislative action. When a- prof essional organization, rather than the 
government, sponsors the regulatory process, it is called certification. 
Certification procedures, are most often used to determine the occupational 
competence of workers, and to identify those who do not meet minimum 
standars; Completiorx of the certification .process advertises a worker's 
abilities to employer, colleagues, and customers. (A few occupations 
offer credentialing through the use of a national registry; qualified 
personnel receive recognition through listing in a -directory available 
to co-workers and employers. This study found no difference between the 
intent or effects of the registration procedure and that of certification.) 
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Certification provides some assurance that a worker has met' certain . 
minimum standards in the 'field. This is not to say that only certifi(^d 
personnel are competent. There are a wide variety of reasons (im^ludiug 
eost^and opposition to the certification concept) why qualified personnel 
^ay choose not to be certified. Nor is the value, of certification con- 
sistent; certification plays a very important role in some fields, ^ile ^ 
in other fields, it has almost no impact on obtaining, maintaining, or . > 
improving empl03mient 'opportunities. . . ' 

It should be remembered that, regardless of its importance, certifi- 
cation is ultimately a voluntary process, while licensing procedures, no- 
matter hdw unrelated to occupational proficiency, are (at least in concept,^ 
if not in enforcement) mandatory. A vast number of licensing procedures 
have developed over the years, some to protect the public welfare, but 
many as revenue producing or registration plrocedures^ or means of 
restricting entry into a particular occupation to the benefit ^of current 
members. During the survey compilation project staff found such wide 
variation in licensing requirements for the same occupation from state to 
state that "necessary occupational competency" (as ascertained by 
licensO^e) cannot possibly be equivalent throughout a licensed trade or 
profession. The study Occupational Licensing ; Principles, and Practices^ M 
deals in depth with questions related to the rvalue of, need for, and 
potential for improvement in the current maze of governmental licensing 
procedures, . . ^ 

Certification is always nationally uniform in character, while state 
licensing (especially but side highly professional occupations-— 5^?^-^ — 
law or medicine) is generally unstandardized and incansi^terfT. While 
licensing is a mandatory process, it was "noF^ thin the scope of the 
project to determine how closely licensing requirements relate to job 
tasks; the disparity among states* requirements suggests that there is 
reason to question that reiajtienship . ^ Although certification procedures 
seem more logical (duevin part to their uniformity) there is some ques- 
tion regarding the'value and necessity of the process. Once again, that 
question lies outside the project's focus. 

TENTATIVE CONCLUSIONS DERIVED FROM THE PROJECT WORK 

Project staff have reviewed a great amount of information in the. 
preparation of the DANTES documents; that review allows the formation of 
several tentative conclusions related to occupational certification and 
licensing. Various references and interviews with a , number of individuals 
and organizations provided ancillary information which helped in* formu- 
lating and substantiating ^ome of . these conclusions^^! t would appear 
that a wide variety of practices and concepts in occupational crederi- 
tialling could benefit from additional evaluation. " 
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With only a few exceptions, independent study courses- do not 
seem to allow completjCon of a comprehensive program of study 
without ^ significant amouht^of additional non-independent 
training, either in" the' form *of supervised job experience or 
ihstitutdLonally-based educational coursework. I-t is generally 
impos'sible to' obtain licensure or certification solely, or 
eveti primarily, through the completion of self-study cburses. - 
Most independent study programs, even' those which result in 
degrees or educational certificates, do not relate dir-ectly to 
any type of licensing or occupational certification program. 
Any attempts to compile listings of these "directly-related" 
courses will be limited, as was the NWREL effort, by the 
apparent small number of existing programs.' With the exception 
of several proprietary school courses, project , staff are 
aware of only one totally appropriate program, the University 
of North Carolina*'s correspondence course in dental- assisting. 
This non-credi't course is approved preparation for the 
American Dental Assistant's Association certification test. A 
few other programs (less than five) are somewhat appropriate 
for consideration! . " 

Considering the va~st. number of occupations, only a §mall number 
have any .type of comJ).etency-ba5e(J examination procedure which 
clearly_evaliiates^a peirsonjs ability to satisfy occupational 
responsibilities. ^*Over 2,000 occupations have' licensing or 
certification procedures of spu^e sor-t^^'but only a ftaction of 
these involve demonstration of the' applicant 's competency 
through 'examination • , ' 

In some occupations (notablv' the^Jbuilding an4 industrial trades) 
and . on some job sites, labor unions haV^e, in a variety of ways, 
greatly influenced job §ntry requi.rements. This employment-- ' 
limiting process is similar in effect to licensing or certifi- ' 
cation procedures. Various readings and interviews suggest 
that unions, are tjrpically in favor of maintaining current 
processes rather than encouraging increased criteria-rrelated 
competency testing as a measure of a worker's slcills. 

The information collected Jjidicates that most states sanction 
educational .institutions %as "licepsing agencies;" graduation 
from. a training institution accredited in a'given field often' 
satisfies a. large portion of the licensing requirements. (As 
.an obvious example, no amount of experience or successful 
-examination performance \d.il allow one to become a physician 
without graduation from'^n'approved medical school. This 
-pr&ce3ure affects many occupations, f rom*^^ law and dentistry to 
barb ering and dry cleaning.)' . 
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Ability , to gain emjployment or advancement in a" field can ^ 
rarely be predicted solely on the basis of satisfying ^ertif i- ; 
cation or licensing qualifications. Credentialling usually 
varies in 'importance, depending on an individual employer, 
. geographic location, job- market, and so on. 

Only a very limited amount of ^research seems to have been done 
in- the area of occupational licensirg and certification. Of^ 
particular cpncern is th&^omplex issue of how closely certi-- 
fication or licensing relates to actual job competency and 
vbat benefits; are derive^ from the existence of these processes. 

Many/Kovernmental agencies and some pf of essioxxal oi^ganizations 
welcome attempts at compiling and examining certification and 
licenslJi^ procedures. Most communications t * the project 
staff were supportive- of the DANTES underta: ^jig. 

Many states have not centralized or even identified occupational 
licensing information in >a msxiner which would laake it relatively 
easy to obtain answers to general or specific licensing respon- 
sibilities; it is far more common to* find anywhere from ten to 
fifty separate departments and boards each maintaining respon- 
sibility 'for one, or several, related occupations. The project' 
staff often had difficulty locating an appropriate information 
source within the state government; it is likely that this- 
situation would create difficulty or inconvenience for those 
seeking assistance with routine licensing problems. 

The certification procedure often seems to be primarily an 
attempt by current members of an occupation to increase the 
professionalism and status of their field. Certification 
procedures are, in almost all cases, initiated from within an 
, occupation rather than 'by any outside legislation or group 
action. Certifying agencies occasionally mentioned the need 
to bring increased recognition- to successful practitioners- of 
an occupation and/or to maintain high standards of competency 
within^ the field. " 

Certification is most predominant in the allied health occupa- 
tions. It is hypothesized that this is partly due to the 
importance and highly tecWcal nature of the work performed 
Cwhich makes both ^'recognition** and ''maintenance of standards" 
viable ratioiiales) * combined with the obvious need to protect 
the public welfare. Certification is a significant part of 
many health fields; salary, advancement potential, and even 
ability to obtain work in the fir ^t place are, because of 
employjer acceptance of the certification concept, often 
dependent-on successfirt completion of tbe credentialling process 



11. State licensing requirements vary so greal^ly that it .is easy 
to question the validity of the criteria on which they at^e • 
based. For example, 1>000 hours of classroom, instruction are 
required for a cosmetology license in Texas, ^ while 1,800 hours 
are required in Arisona. Does this imply that the cosmetolo-^ 
gists in Arizona are considerably better ^qualif iaS,''' or. that^ ' 
the training in Texas is significantly more effective? More 

^ . likely it . suggests that the requirements themselves are 

^ somewhat arbitrary, that they are not the .r^esult of a system^ 
•atic pairing of the skills with the curridulum. Information . 
collected for the survey revealed a multitude of such discre- 
* ■ ^ pancies; in only a few licensed occupations was there 

substantial uniformity among state licensing requirements. 

12. ^ Information collected suggests that licensing- and certif icatton 
/ tests may not generally be constructed under the s^e' rigid ^ 

/ ' criteria which would be used for educational tests of 

equivalent importance. In all the test descriptions examined, 

w project staff found no mention of validity or xeliability 

..checks.. T^t prpcedures were often not standard izedv. Perhaps 
most significant wa.s the fact that testing in many occupations*^ 

, was of the norm-reierenced type in which a candidate ^nust;' 
^. • obtain a. percentile score which- exceeds a specified "passing" ' 

. score before being granted the credentials^ These cutoffs 
frequently seemed unrelated to any competency-based criteria 
and appeared extremely arbitrary. Several occupational 
examination$ have been developed Under Vigorous criteria that 
help ^ensure their appropriateness^ and value. Yet many .other 
tests do not necess^arily succeed in determining the tfue^ 

^competence of the candidat-e. They are seldom rigorously* 
constructed tests with a strong correlation to a proven set c 
^ . of essential abilities. Many appear to have been established 
solely ^as a result of legislative pressures, or the opinions 
of members of the occupation' regarding how much a.. person should 
khow^ about what ^topics. Such persons hold informed opinions, 
to be sur^, but tests constructed without empirical justifica- 
tion for their content may -be considered a .significant weakness 
in the overall credentialling process. 

This is in no way meant to imply. that these tests are 
inappropriate* — ^eir appropriateness is a question which 
requires extensive/research before . even tentative conclusions 
can be drawn. Nor is it meant to suggest •that the test 
developers were not qualifiedV Tl^e implication is simply that 
few licensing, or" certification agencies, can empirically 
demonstrate that their credentialling tests do ^indeed differ- 
entiate between individual^ on the basis of occupational 
competency. 

- ' ■ 12 .. 
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In conclusion, this, proj ect has gathered . a wide variety of' informa- 
tion in the field of occupational licensing and certification. Yet 
because this area has not been'^^^tensively investigated,- additional 
research would seem to be in order, ' • ■ ^ : " 
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